Rastervision is an open source framework that uses python to build computer vision models on satelite imagery. Our group wanted to use drone imagery and high resolution satelite imagery to segment fairy fungus rings in cranberry bogs and urban tree canopy cover respectively. We used an AWS instance to create a linux environment with rastervision and docker installed. Our group had the following goals: run rastervision quickstart, run rastervision example(s), run rastervision on personal data. Our group was only able to complete the first 2 objectives however we hope to provide information on how to prepare data to run your own semantic segmentation based on personal data. Expertise barriers, model choice, and calibration are all limitations in earth observation identified by (the previous class?). Rastervision attempts to act as a potential solution to these limitations by streamlining the model choice, making deep learning algorithms and computer vision accessable to non computer scientists (like geographers). The two models that rastervision uses to run its different examples are Mobilenet (semantic segmentation) and Resnet50 (chip classification). These models use deep neural networks to train computers to recognize pixels in the case of Mobilenet and objects in the case of Resnet50. Rastervision allows these complex computer vision models to be used by inexperienced users to perform semantic segmentation and chip classification on large datasets. However, this is not necessarily a perfect solution to the limitations of earth observation as users do not a strong grasp of how the models run which will influence the outputs. The push pull between making models like Mobilenet acceccessable and understanding the algorithm is always going to be balance. Rastervisions is another useful stepping stone in advancing the earth observation.
What were the methods you developed and/or applied? With the help of Dr. Lyndon Estes and Lewis Fishgold, we were able to run rastervision in a GPU p3.2xlarge ubuntu linux instance on Amazon Web Services (AWS). The instance was created from an image with Docker and NVidia installed. Cygwin was used to access the instance. SSH compatability and python were two tool options necessary to download within cygwin. Once this was installed we needed to install the AWS command line tools using:
pip install awscli
The next step was to produce Public SSH keys so multiple users could access the AWS instance. Within cygwin, we followed the first two steps described in Digital Ocean and then to set security for our permissions followed up with:
chmod 400 ~/.ssh/id_rsaWe had success running the ISPRS Potsdam Semantic Segmentation example from rastervision examples github. The input data is below.
Figure 1. Borrowed from the ISPRS 2D Semantic Labeling Contest - Potsdam webpage.
This example uses three different data layers in order to run the predictions. The data is 5cm aerial imagery which uses RBGIR bands to create a true color composite. The second layer is a normalized DSM using lidar imagery. Finally, ground truth labels are created by hand for 6 classes: building, tree, low vegetation, impervious, car, and clutter. The format of six classes was ideal as the files utlized 3 bands, each with a binary value, for a total of 6 possible combinations. The RGBIR and label files were available as TIFFs. The lidar data was only used for distinguishing the classes and was not necessary beyond identifying the training data. Therefore, for future imitations lidar data is unneccesary if other means of classifying training data is used. All of the training data was downloaded through the request form indicated in the raster vision examples github, and uploaded to an Amazon S3.
After the initial set up, and our Public SSH keys were added by the instance administrator, Lyndon Estes, we were able to log into the instance in cygwin via:
<AWS USER NAME> ubuntu@<INSTANCE IP ADDRESS>
Once there, our successful workflow was:
docker system prune -a
cd raster-vision-examples
vim scripts/console
Then to ensure the GPU is referenced, edit line 42 to:
IMAGE=raster-vision-examples-gpu
Additionally, edit line 34 to resemble:
docker run --runtime=nvidia ${NAME} --rm -it ${TENSORBOARD} ${AWS} ${RV_CONFIG}
Once those adjustments are done, use “q” to exit to get back to the root. Then we can proceed to set up and run the script.
cd raster-vision-examples
scripts/build --gpu
scripts/console
rastervision run local -e potsdam.semantic_segmentation -a root_uri <Path to S3 data ouput location> -a data_uri <Path to S3 Data Folder>
Figure 2. Predicted image after 2% completion.
Figure 3. Original RGBIR image.
Figure 5. Output image of completed rastervision prediction.
The goal is to mimick the ISPRS Potsdam training data format to substitute our own data to use with rastervision. An orthomosaic was created in PixD from multispectral data collected in NIR, Green, and Blue flown by an unmanned aerial vehicle in 2015 over a cranberry bog. The cranberry bog is under treatment for an infestation of fairy ring fungus that is visible from the aerial images. Traditional classification methods have not been sensitive enough to distinguish between healthy, effected, and recovering vegetation within the bog and the hope is rastervision will provide more realistic results than the training data can provide. Unlike the Potsdam training data, in order to create a labels TIFF, a specific test field was chosen and an Iso Cluster Unsupervised Classification was run for 6 classes (Figure 6). Though our interest does not necessarily require 6 classes, this is following the lead of what was available within the Potsdam dataset and would also allow opportunity for sparsing of traits such as bare soil, multiple levels of recovering vegetation, and shadows due to clouds which are visible in the orthomosaic. To create individual images, a fishnet was laid on the test field (Figure 7).